Typically, in most computers, user interface (UI) component selection is done by dragging mouse icon across over to the UI component of interest and then clicking the mouse button, indicating that the selection is taking place. Alternatively, UI selection is performed with a touch pad and a finger or joystick of some kind or other methods, where physical action is performed and is translated into UI component selection.
In a computer with a touch screen, UI selection is haptic and is driven by a hand or finger touch, which is detectable by the touch screen. UI component is selected by touching it. Some systems use helper stylus devices which also fall into the haptic class of devices.
In many AR (Augmented Reality) or VR (Virtual Reality) head mounted solutions, selection of the projected UI components is performed by tracking hands of the user for example.
With currently emergent AR technologies, hand movement is tracked by variety of color or depth sensors and hand movement is used as a controlling mechanism to select UI component or to signal and trigger system action. With the orthodox AR and VR systems, gestures are used as system commands or interpreted by specific application in context of which UI selection is performed by way of an external bodily action. Such action is then correlated to 2D or 3D UI geometry where selector UI component (arrow/hand) corresponding to the selector moves over the UI component on the screen, triggering action signals to the system to select the UI component. That in turn triggers an event that is sent by the system to the system/application being selected. The application then receives the trigger signal and initiates an appropriate response action.
With the orthodox AR and VR systems, gestures may be used for selection without selector UI component (arrow/hand), simply by the correlating geometry of the selecting body part (hand or finger, etc.) with the 2D or 3D geometry of the UI interface in the given AR or VR system. As soon as focus of the of the gesture points to a specific component the component is selected.
In systems of smart contact lenses with an embedded display, UI components may be depicted on the contact lens embedded display. Providing ability to select UI component out of many and thereby triggering system action as a way for the user to interact with AR or VR enabled smart contact lens is critical to making smart contact lens useful and practical. However, there are certain physiological and anatomical limitations of the human ocular system that need to be accounted for.
Generally, human eye lets light reflected from an object to travel through a lens, there to hit retina that is a light sensitive surface located at the back of the human eye. The retina of the eye may generally be divided into several discreet sections that are differentiated by a level of their respective light sensitivities. For instance, the retina may be divided into a fovea and a peripheral area, wherein the peripheral area further includes a parafovea belt, circumscribing fovea, and a perifovea outer region further from the center of the peripheral area, circumscribing parafovea. Retina mostly consists of two types of photoreceptor cells: cones for daytime colored perception and rods for dim light and black-and-white vision. In human eye, there are approximately 7 million cones and in the range of 75 to 150 million rods. Fovea, which is at the center of the retina consists of mostly cones so allows for good quality colored vision. The fovea enables a clear, a sharply focused, and a colored vision, whereas the peripheral area is used for a low light vision, for detection of movements and for discerning between different colors and shapes.
A sharp vision is possible due to a foveal area which is situated at the bottom of the eye directly in front of the lens of the human eye. However, the foveal area represents only some degrees of visual angle that enables a person to see a very limited portion of the observed view in sharp focus. Further, a peripheral vision is critically important and plays a crucial role in visual perception. Brain registers and processes information that falls into the region of the foveal area as well as information from the peripheral area. Initial and partial visual data acquisition is performed via peripheral view and for full and detailed data acquisition, eye moves to bring information of interest into focus; that is to sense an image in focus with foveal area.
Normally, in conventional vision systems, whenever an image is displayed in front of the user, such as on TV, a tablet or any other head mounted displays, text on the paper, location of the view remains constant and only eyes gaze direction changes relative to a stationary part of the image on the display to focus on other portions of the display in order to have full image data ingestion.
However, provided, a transparent or a semi-transparent or a non-transparent display is embedded into the contact lens, an image or a video being superimposed onto the real world objects view, in front of the user, may be displayed. Such kind of an embedded display is naturally, spatially associated with and locked in, relative to a position of the human eye. Because the embedded display will be shifting with every movement of the eye, only a part of the image being present at the center of the embedded display, would be in sharp focus and a user will not be able to perceive other parts of the superimposed image in clear focus. Further, the human eye position adjustments will not enable the eye to see other parts of the image, in focus because the embedded display moves with the movement of the human eye and an image disposition on display, a priori, does not change.
Furthermore, information about real world objects present in front of the user is superimposed onto the embedded display in spatial respect to geometry of objects visible to the user. For instance, an Augmented Reality (AR) enabled application recognizes real world objects, determines contextual annotations/descriptions associated with the recognized real world objects and superimposes onto the embedded display, information related to the real world objects in such a way that the information is overlaid near or over the real world objects in the view. In such an application, the information which will be a virtual object that would appear stationary relative to specific surrounding environment of the user. In another exemplary embodiment, by application of a display integrated into contact lens, information may be superimposed on the display irrespective of the spatial position of objects in the view of an observer. For example, text (article, book, etc. . . . ), dashboard, with temperature, time, stock quotes, running news line and other information. This data is independent of what the user sees via the contact lens. In another exemplary, application of display integrated into the contact lens is Virtual Reality (VR), wherein the view presented to the user is semi-transparent or non transparent, creating virtual reality view. There is multitude of other applications of an embedded display and any use of contact lens based embedded display will present the same limitation as described above.
To overcome the aforementioned problem, image on the embedded display may be shown at a center position in order to make an entire image in focus. In order to display the entire image at the center position and in focus, the image needs to appear as being far away from a user. However, this approach presents a number of limitations, such as: 1) amount of information and image size displayed in focus at “far away” is minimal, and 2) there is no peripheral view available which further limits usefulness of such an approach.